Tomáš Pospíšek's Notizblock
Rationale
Since I was suspecting aqbanking to be skipping some entries in SWIFT MT940 documents when importing them, I wanted to have a close look at what libaqbanking's parser was doing.
Scope
I am running a Debian system, so this blog entry is documenting the process from a Debian perspective - however it should be possible to adapt the steps taken here quite easily to other distributions and - to debugging of of shared C libraries in general.
Reducing the code size: Command line
Libaqbanking is used by Gnucash and a couple of other programs to import MT940 documents, however, thankfully, there's also a command line utility to do the same, which allows to reduce the problem to a smaller set of tools.
In Debian you can:
apt-get install aqbanking-tools
and then, to parse a document:
Importing a MT940 document with Aqbanking
aqbanking-cli import --infile=/path/to/MT940_123456789_P_2014123112345678.940 --importer=swift --profile=SWIFT-MT940
Aqbanking documentation
You can call
aqbanking-cli --help
aqbanking-cli import --help
to get some information about the usage of aqbanking-cli but the info there is not comprehensive.
There is a bit more information in the Aqbanking handboook.
The latter tells us that you can get debugging information from the aqbanking-cli tool like this:
export AQBANKING_LOGLEVEL=debug
export GWEN_LOGLEVEL=debug
aqbanking-cli import --infile=...etc...
Getting the source code
In Debian you can get the source code for some package you are running by issuing:
apt-get source libaqbanking34
For that to work, you need to have the relevant source URL in /etc/apt/sources.list:
deb-src http://ftp.ch.debian.org/debian/ jessie main
You can also get all the headers and tools that building the package depends on via:
apt-get build-deps libaqbanking34
This will ensure, that you are working with the source code that corresponds to the tools you are actually running - as opposed to newer upstream code. Of course you need to consider running upstream code, since it might contain the fix for your problem already and also if you want to contribute changes back you should target current upstream code.
Debuggers
Since running aqbanking-cli doesn't give me the level of detail I want, I proceed to a debugger.
I've tried the following gdb frontends:
ddd - extremely powerful and comprehensive, but alas, like has been the case for the last dozen of years, very crash prone and thus not really usable.
kdbg - although setting breakpoints in the library, gdb would not stop at the breakpoint.
cgdb - a console/ncurses interface. Use 'Esc' to jump to the source window, 'i' to jump back to the gdb prompt, 'Space' to un/set breakpoints. Cgdb does display the source code, but is not able to display variables. The geat advantage is the direct access to the gdb prompt: setting pending break points is possible and gdb actually will stop there.
nemiver - seems to suffer from my perceived 'protect the idiot user from details' Gnome paradigm. It f.ex. doesn't seem to be possible to set paths in a textual fashion - one always needs to click through file hierarchies in the file chooser. Additionally the file chooser seems to be inspecting the presented files, so picking the executable takes a minute or two. It also has the 'too intelligent, but not intelligent enough to let go' problem in that when you run the executable you get prompted half a dozen times to provide nemiver with the source code to the executable. On the other hand, nemiver provides access to modern gdb features such as breaking on entering named functions, which we need. And it also stops execution at the beginning of main() automatically, which in the past was always a WTF moment for me - why would I start a debugger in the first place, if it won't stop when running the program? In spite of the ambiguous Gnome philosophy, nemiver seems to be stable, and allows to set the breakpoint in the library which is what I need.
De-optimizing compiled code for debugging
Debugging the library has the problem that when stepping through the code the debugger seems to be randomly jumping through the code without any apparent reason. The superficial reason is that code gets optimized by the compiler, and the mapping between generated code and lines in a source file gets shaky. However for the library at hand it is not clear to me that the mapping has to be that erratic and random as not to make any sense at all whatsoever.
Therefore we need to de-optimize the produced code. This can be achieved by setting the compiler flags -O0 and -g3. Unfortunately, CFLAGS are set in about 80 Makefiles throughout the library code, which makes changing them by hand cumbersome.
In principle it should be possible to set them via Debian means like this:
DEB_BUILD_OPTIONS="noopt,-g3" dpkg-buildpackage -rfakeroot
But for some reason this didn't seem to get correctly transported through to into the build process.
So what I did instead was:
CFLAGS="-g3 -O0 -fstack-protector-strong -Wformat -Werror=format-security -Wall -Wdeclaration-after-statement" ./configure --with-backends="aqnone aqhbci aqofxconnect aqebics"
borrowing part of the flags from Debian's debian/rules build file and a part from aqbanking's Makefile. And then:
make
This would produce a correctly de-optimized library.
Using the de-optimzed library while debugging
Parsers for different formats are aqbanking plugins, and live under:
/usr/lib/aqbanking/plugins/34/dbio/
In principle it should be possible to load your own libraries by setting LD_LIBRARY_PATH - however I wasn't sure whether aqbanking is using its own method of loading plugins and thus what I did was to replace:
/usr/lib/aqbanking/plugins/34/dbio/swift.so
with the version I've just compiled, which I retrieved from:
$LIBAQBANKING_SOURCE_DIR/src/plugins/parsers/swift/.libs/swift.so
Using pending break points
When using vanilla gdb to debug the library you'll want to set a breakpoint inside that library. But, since at the moment of execution the library is not loaded yet, gdb will not be able to find the code in question. In order to let gdb know that it only should set the break point once the library gets loaded, you can do the following:
set breakpoint pending on
directory $LIBAQBANKING_SOURCE_DIR/src/plugins/parsers/swift
break AHB_SWIFT_Import
In combination I need to set the following in gdb:
set env AQBANKING_LOGLEVEL debug
set env GWEN_LOGLEVEL debug
file /usr/bin/aqbanking-cli
directory $LIBAQBANKING_SOURCE_DIR/src/plugins/parsers/swift
set breakpoint pending on
b AHB_SWIFT_Import
cd $LIBAQBANKING_SOURCE_DIR
set args import --infile=/path/to/MT940_123456789_P_2014123112345678.940 --importer=swift --profile=SWIFT-MT940
Thanks
I hope this will help you debug your shared library problem.
Tomáš Pospíšek, 2015-02-27
